Convolution, Smoothing, and Image Derivatives

Computer vision operates on images that usually come in the form of arrays of pixel values. These values are invariably affected by noise, so it is useful to clean the images somewhat by an operation, called smoothing, that replaces each pixel by a linear combination of some of its neighbors. Smoothing reduces the effects of noise, but blurs the image. In the case of noise suppression, blurring is an undesired effect. In other applications, when it is desired to emphasize slow spatial variations over abrupt changes, blurring is beneficial. In yet another set of circumstances, these abrupt changes are themselves of interest, and then one would like to apply an operator that is in some sense complementary to smoothing (in signal processing, this operator would be called a high-pass filter). Fortunately, all these operations take the form of what is called a convolution. This note introduces the concept of convolution in a simplistic but useful way. Smoothing is subsequently treated as an important special case. While an image is an array of pixel values, it is often useful to regard it as a sampling of an underlying continuous function of spatial coordinates. This function is the brightness of light impinging onto the camera sensor, before this brightness is measured and sampled by the individual sensor elements. Partial derivatives of this continuous function can be used to measure the extent and direction of edges, that is, abrupt changes of image brightness that occur along curves in the image plane. Derivatives, or rather their estimates, can again be cast as convolution operators. The next section uses a naive version of differentiation to motivate convolution. The last section of this note shows how derivatives are estimated more accurately.